MADRE: The Memory-Aware Data Redistribution Engine
نویسندگان
چکیده
We report on the development of a new computational framework for efficiently carrying out parallel data redistribution in a limited memory environment. This new library, MADRE (The Memory-Aware Data Redistribution Engine), is an open-source, C/MPI-based toolkit designed for quick and easy integration into application codes that have demanding data migration needs. At the same time, MADRE exposes a lower-level API that greatly facilitates the development and incorporation of new algorithms into the MADRE framework, thus serving as a potential organizing entity for continued research in this area. Finally, we develop, describe, and test in detail several new parallel redistribution algorithms that are incorporated into the
منابع مشابه
A Memory-Efficient Data Redistribution Algorithm
Many memory-bound distributed applications require frequent redistribution of data. Pinar and Hendrickson investigated two families of memory-limited redistribution algorithms. The first family has many advantages, but fails on certain inputs, and, if not implemented carefully, may lead to an explosion in the number of local data copies. The second family eliminates the possibility of failure a...
متن کاملERIS: A NUMA-Aware In-Memory Storage Engine for Analytical Workload
The ever-growing demand for more computing power forces hardware vendors to put an increasing number of multiprocessors into a single server system, which usually exhibits a non-uniform memory access (NUMA). In-memory database systems running on NUMA platforms face several issues such as the increased latency and the decreased bandwidth when accessing remote main memory. To cope with these NUMA...
متن کاملA Transparent Runtime Data Distribution Engine for OpenMP
This paper makes two important contributions. First, the paper investigates the performance implications of data placement in OpenMP programs running on modern NUMA multiprocessors. Data locality and minimization of the rate of remote memory accesses are critical for sustaining high performance on these systems. We show that due to the low remote-to-local memory access latency ratio of contempo...
متن کاملBuilding Disclosure Risk Aware Query Optimizers for Relational Databases
Many DBMS products in the market provide built in encryption support to deal with the security concerns of the organizations. This solution is quite effective in preventing data leakage from compromised/stolen storage devices. However, recent studies show that a significant part of the leaked records have been done so by using specialized malwares that can access the main memory of systems. The...
متن کاملAutomated Analysis of Pressure Build up Tests Affected by Phase Redistribution
Analytical Solutions and type curves for the constant rate radial flow of fluid in both conventional and naturally fractured reservoirs including the effect of wellbore phase redistribution are presented. An automated procedure for non-linear least square minimization using the analytical solutions and their derivatives with respect to the unknown parameters developed to analyze the pressur...
متن کامل